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Teaching Online: STEM Education in the Time of COVID, Bay View Analytics 
is released under a Creative Commons Attribution 4.0 International (CC BY 4.0). 
Report available at http://www.bayviewanalytics.com/. 


You are free to: 


Share — copy and redistribute the material in any medium or format 
Adapt — remix, transform, and build upon the material for any purpose, even commercially. 


Under the following terms: 
Attribution — You must give appropriate credit, provide a link to the license, and indicate if you made changes. 


You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your 
use. 


There is NO requirement to ask permission to use Bay View Analytics / Babson Survey Research Group 
publications under the CC BY 4.0 license. While it is not a requirement, we always like to hear how you are 
using our material. 


You may reproduce the report in any media (e.g., print or electronic distribution) you wish and distribute in any 
way you want to. We do ask that if you are providing a link to the full report PDF version that you use the link 
on our web site rather than hosting it on a different site, this allows us to track the level of interest in our 
reports. 


We understand that some publications may require a explicit statement that the intended use is approved. If a 
copy of this page is not sufficient for such purposes, please let us know, and we will provide the required 
documentation. 


\N ; : STEM Education in the Time of COVID 
AA Bay View Analytics 2 


ACKNOWLEDGMENTS 


Bay View Analytics, formerly known as the Babson Survey Research Group, 
was created at Babson College in 2004. Our relationship with Babson College 
spanned over a hundred reports and several million downloads; they were a 
tremendous partner. 


No survey report is possible without the willing cooperation of the 
respondents. We wish to thank the many faculty members who took the time 
to provide us with their detailed and thoughtful responses amid a global 
pandemic. We know that you are very busy, and we appreciate your time and 
effort. 


Special thanks to all the respondents who provided us with such detailed and 
useful comments. Many also provided permission for these comments to be 
quoted in the report — this commentary offers a rich texture to the results that 
would not otherwise be possible. 


We want to thank our sponsors: Carolina Distance Learning, HHMI 
Biointeractive, Every Learner Everywhere, DigitalEd, and the Online Learning 
Consortium. Without their financial support and expertise, this survey would 
not be possible. 


Research like this is a team effort. It has been a pleasure working with 
National Survey on the State of Online STEM Education (NSSOSE). In particular, 
the many conversations with Devon Cancilla, Kate Lee-McCarthy, and Lynette 
O'Keefe throughout this process were invaluable in this effort's success. Kate's 
and Lynette's tireless support as OLC project liaisons allowed us to design, 
test, and gather data through activities such as the OLC STEM Summit and OLC 
IDEATE, which were essential to the overall success and richness of the 
project. 


Finally, we want to thank our readers. We continue to receive a steady stream of 
feedback on expanding and improving our reports, and this report is better for 
your input. Please continue to let us know how we can continue to improve. 


Jeff Seaman 

|. Elaine Allen 
Nate Ralph 

Bay View Analytics 
2021 


\N ; : STEM Education in the Time of COVID 
AA Bay View Analytics 3 


EXECUTIVE SUMMARY 


The COVID-19 pandemic sparked a disruptive shift from face-to-face 
instruction to distance-learning, impacting academia in the U.S. across 
undergraduate and graduate levels. This study, conducted in the fall of 2020, 
explores the impact these changes have had on higher education faculty in 
STEM fields. 


Findings include: 


e Restrictions on gatherings and similar safety requirements have 
forced faculty to shift to online instruction in unprecedented 
numbers. 


e 73% of respondents surveyed report having converted face-to- 
face coursework to remote learning, despite more than a third 
of respondents having no prior experience with online 
education. 


e The most serious barrier to successfully implementing online 
STEM education, as reported by respondents, is a perceived 
lack of student motivation to engage with online coursework. 


e While generally optimistic about the potential effectiveness of 
online labs, only 18% of survey respondents have made use of 
them in their coursework. 


e Respondents believe that the transition to online-learning has 
exacerbated the divide between the have and the have-nots, as 
a lack of resources (including but not limited to access to 
technology, robust internet connectivity, and support services) 
unfairly impacts underserved student populations. 


e The findings highlight the need for additional research into the 
effectiveness of online STEM education. 
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INTRODUCTION 


Background 


Online education has seen extraordinary growth over the last decade.? 
Academic and non-academic institutions alike are producing an extensive 
number of online courses and programs, incorporating an expansive array of 
eLearning tools and strategies. 


Despite this growth, academic disciplines have varied in their acceptance and 
incorporation of online learning into their educational portfolios. There is no 
hard data on STEM enrollments in distance programs, so we can only 
speculate on how large an issue this is for STEM. Even without the data, there 
exists among some a perception that STEM does lag other disciplines, 
reinforced when comparing types of presentations at national conferences — 
consider the Online Learning Consortium’s Accelerate conference, where non- 
STEM related presentations significantly outnumber those within STEM fields. 


Are these perceptions reflected in STEM faculty's beliefs about the efficacy of 
online education, or is there something inherent within the STEM disciplines 
(such as the need for labs) that makes online education impractical? What 
online teaching experiences have STEM faculty had to form their opinions and 
actions about online education? This survey, Teaching Online: STEM 
Education in the Time of COVID, has been designed to investigate these 
questions and identify real and perceived barriers to moving STEM education 
online. 


Initially scheduled to be conducted in March of 2020, most institutions’ 
emergency response to deliver their programs online due to the growing 
COVID-19 pandemic required a tectonic shift in the survey's design and its 
underlying assumptions. The pandemic-related experience of faculty — many 
required to teach online for the first time — would dramatically inform their 
opinions. In light of this, the survey team revamped the survey to better 
reflect the emergency move to online teaching, and to see how the COVID-19 
pandemic has informed opinions about online education, and the future of 
online STEM education. 


" https://nces.ed.gov/programs/digest/d|8/tables/dt!8_311.22.asp 
nnn nnn nT 
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STUDY RESULTS 


Description of Respondents 


The results presented in this report are based on 896 responses from STEM 
(Science, Technology, Engineering, and Mathematics) faculty teaching at the 
graduate and undergraduate level.” Data collection was conducted during the 
last week of October, 2020. Respondents represented multiple STEM 
disciplines, with faculty from the biological sciences making up the single 


largest group, followed by engineering, and those teaching in the physical 
sciences. 


Discipline 


g Biological Science 

mw Engineering 

mw Physical Science 

mg Computer and 
Information 


gw Math and Statistics 


g Other 


? Details on the sample used are provided in the Methods section of this report. 
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The majority of respondents were male, though there is a strong 
representation of female STEM faculty. Very few respondents — a total 
amounting to less than a percentage point — identify as Non-binary. 


Gender 


# Male 


m Female 


A third of the respondents surveyed have been teaching between one to ten 
years. Another third of those surveyed have taught for between 11 and 20 
years, while the remaining third have more than 20 years of teaching 
experience. Of the responses collected, 14% were from adjunct faculty. 


Number of years teaching Adjunct status 


ikea m Adjunct 
14% 
miito20 
wg Not Adjunct 
@ 21 to 30 
@ More than 30 
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Online Teaching Experience 


Faculty Voices: 


“For math, sharing an iPad screen, using video, and recording lectures, not much is lost 
compared to lecturing in person.” - Mathematics Faculty 


“Given where we started and the unique aspects in STEM, it is amazing what has been 
done and there is so much potential for what we can do.” - Engineering Faculty 


“STEM courses are best taught in the classroom as many of my students have 
confessed. The online experience cannot replace what is done in a face-to-face, 
hands-on class.” - Computer and Information Sciences Faculty 


More than a third of respondents had no online experience prior to the 
COVID-19 pandemic. Just over 40% report having taught an online course, 
while 37% report having developed one. 


Online experience prior to pandemic 


Taught an online course | 8% | 
Developed an online course | 87% | 
Taken an online course | 34% 
Other experience im 6% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Wy F : STEM Education in the Time of COVID 
AA Bay View Analytics 8 


Faculty Voices: 


“| think we will get better at teaching STEM online with more experience and support.” 
- Mathematics Faculty 


“As we learn to deliver content online, our students will become better online 
learners.” - Mathematics Faculty 


“What makes education valuable is the direct fresh contact with faculty.” - 
Mathematics Faculty 


“It isa *lot* more work to get everything where it should be and! am not sure if that is 
helping my students learn.” - Engineering Faculty 


“| feel that teaching online has significantly hindered effective interaction between 
teacher and student, as well as between students, which is an important aspect of 
learning new material. Students aren't learning as well or understanding material 
as deeply. Not exactly sure why, but I've seen lower scores and a much bigger 
dispersion in scores.” - Mathematics Faculty 


These positions transformed dramatically with the onset of the pandemic. The 
proportion of respondents with no online experience dropped to just 4%, with 
73% of respondents stating that they’ve converted their courses to emergency 
remote learning. 


Online experience since the pandemic 


Taught an online course | 22% | 


Other experience ial 5% 


None E 4% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
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Faculty Voices: 


“ like that this new way of doing things helps some students. They can log in and re- 
watch my lectures which is a big positive (my daughter is a University student so | 
have also seen the benefit of this first hand). | dislike how it presents a big challenge 
for some due to learning style and other issues, for example for students who don't 
have good internet and/or a home environment that supports learning.” - 
Engineering Faculty 


“It is very hard to teach when looking at a screen filled with names; many or all 
students do not turn their video on. | can sympathize with them (it is hard to stare 
at a screen the whole day), but it also makes it hard to interact: they see you but 
you do not see them. :)” - Mathematics Faculty 


“My students are succeeding in the online environment. They love the remote 
synchronous format.” - Mathematics Faculty 


“Collaboration of materials between other instructors in my field (geology) has been 
the most important for the development of my lab classes.” - Physical Sciences 
Faculty 


“For computer programming the on-line experience has some advantages, particularly 
the ability to quickly do screen sharing.” - Computer and Information Sciences 
Faculty 


Ninety percent of the respondents surveyed were teaching one or more STEM 
courses for the Fall 2020 term. A further third were teaching two courses, 
while 17% and 15% of respondents were teaching three and four courses, 
respectively. Teaching multiple STEM courses in a single term may impact the 
perceptions of online education due to the additional workload of converting 
multiple classes online, as opposed to just one. 


Number of STEM courses teaching this term 
mw None 
gl 
m2 
m3 


m 4 or more 
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Faculty Voices: 


“Everybody (students, faculty, staff, parents) hates online education, and are anxious 
to return to a normal, traditional college education process. | think on that we can 
all agree. But as | said before, we can't do that until there's a clear end to this 
worldwide pandemic. So, we're all doing the best we can with the options we have 
available to us.” - Engineering Faculty 


“It's tough, especially exams etc., but actually gives the opportunity to some shy 
students to chat and ask questions. The same students would never raise their hand 
and ask in front of a huge class. That's the only good thing | have noticed.” - 
Physical Sciences Faculty 


‘It is a challenge but with effort and creativity | can make it work at some level, maybe 
not as well as in person but good enough for the time being (not long term 
though).” - Biological and Biomedical Sciences Faculty 


Respondents were overwhelmingly teaching at the undergraduate level, with 
just 30% teaching courses at the graduate-level. 


Level of course 


Graduate level a 


Other i 3% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


: d STEM Education in the Time of COVID 
NN Bay View Analytics 1 


Faculty Voices: 


“STEM courses, specifically laboratory courses, provide a subpar education when 
taught online with no hands-on experience.” - Biological and Biomedical Sciences 
Faculty 


“Zoom has been a savior. | made my students spend about S60 on hardware because 
they need to do the lab at home. That means then scheduling zoom demos to see 
they can build it and written reports and reading all their code...this is with the idea 
if they are going to cheat when we evaluate all 3 of those results we are more likely 
to catch them.” - Engineering Faculty 


“Re-designing hands-on work so that it can be done in-home often means dumbing it 
down a little, but | find the work still has merit. Everyone (student, prof, other) just 
has to lower their expectations a bit around the professionalism aspects.” - 
Engineering Faculty 


“The use of the online whiteboard does not offer the same impact as using an in- 
person whiteboard. Students say there is sometimes a delay and | am unsure if that 
is a technological issue on my end or theirs.” - Mathematics Faculty 


In a large departure from their previous experience, only 17% of the 
respondents were currently teaching a face-to-face course. 


Current teaching modality 


Online 
Blended/Hybrid 


Face-to-face 


Other U 4% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Viewed as whole, these results tell the story of a hurried transition for higher 
education STEM faculty and their charges, as they were forced to dive 
headfirst into the world of remote instruction with little prior experience. 
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Barriers to Online STEM 


One of the first steps that the National Survey on the State of Online STEM 
Education (NSSOSE) took to understand what faculty perceived as the most 
critical barriers to providing online STEM education was the half-day OLC 
Accelerate 2019 STEM Research Summit. The participants, STEM faculty and 
administrators, were queried about which barriers might prevent wide 
adoption of online STEM education. A participant survey before the workshop 
was used to get initial feedback on what faculty and administrators believed 
were the most critical issues. These were discussed in depth during the 
session, and followed up with a post-workshop survey to get further validation 
of the issues. All of this was prior to the pandemic. 


The emergence of the pandemic, and the experience of many STEM faculty 
moving their courses to emergency remote learning in the Spring of 2020, may 
have changed faculty views. A test survey was conducted at the end of spring 
term 2020, and while it was clear that many barriers identified pre-pandemic 
were unchanged, many faculty had reconsidered the issues. 


Additional guidance on identifying the revised view of barriers to online STEM 
education came from the OLC Ideate Labs for Online STEM: Innovating STEM 
Education, August 19-21, 2020. The topics listed as potential barriers was 
revised based on a combination of a pre-survey, and online sessions during 
the conference, as well as a follow-up post conference survey of the 
participants. A few items were dropped, a few new ones added, and several 
had wording changes based on this feedback. The final questions used for this 
survey include a dozen potential barriers listed, each coming directly from 
faculty suggestions, as well as an open-ended response where any other 
issues could be added. 


It is important to keep the timing of these responses in mind. Faculty were 
responding to the question about barriers while in the middle of a term where 
most of them were teaching a course online out of necessity. Faculty did not 
have the luxury of time to design the best teaching approach, so their 
perception of the barriers is flavored by this experience. 
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Faculty views on barriers they faced moving STEM courses online were varied. 
Only one issue (academic integrity) was selected by a majority of the 
respondents. Only 9% of the respondents reported that they did not think 
there were any significant barriers. 


Open-ended responses were added by 14% of the respondents. Several 
themes emerged from these free-text responses. The most common theme 
was a feeling of being disconnected from students, and missing the person-to- 
person interaction of their face-to-face course. The second most-common 
reply was a feeling that the online education experience was "not as good," or 
that "in-person delivery is more effective." 


Many faculty also used this option to expand of the choices they made from 
the list of barriers, including details on their worries about academic integrity 
and their inability to get students engaged (or even to attend classes). 


Barriers to online STEM education 


Inability to ensure academic integrity 

Students not sufficiently motivated 

Equity issues for some of my students 

Student internet and computer access 

Inadequate online laboratories 

Students not technologically prepared 
Lower retention rates in online... 


Support developing or delivering... 


e 
& 
e 


Other 
Institutional support for students a 12% 
Lack of technological support Ee 10% 
No significant barriers ea 9% 
Lack of technology al 5% 
Online courses will not transfer | 2% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
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Faculty Voices: 


“If we cannot deliver engaging content, retention will be difficult, which impacts 
enrollment and funding of the institution.” - Physical Sciences Faculty 


“It is a very difficult situation right now. Although | provide as many interactive things 
as! can, my students still will not engage. | have between 20 and 30 who attend my 
live class, and only 6 or 7 will interact with me. | have tried many things to get them 
to interact and open up, but they will not.” - Mathematics Faculty 


On average, faculty members selected three or four barriers from the list. All 
faculty who selected more than one barrier were asked which of the barriers 
was the most serious. The most common cited barrier in this ranking was that 
"Students are not sufficiently motivated to succeed in online courses," 
selected by just under one-quarter of all respondents. Respondents cite 
instances of students that don’t show up to lectures, shy students who are 
reluctant to participate or activate their webcams, and students who leave a 
lecture early, as supporting evidence for this belief. 


Most serious barrier 


mw Students not motivated 
mw Academic integrity 

m Equity issues 

m Online laboratories 

m Student access 

m@ Lower retention rates 


mw All other 


For all of their concern with student engagement, faculty do not put all the 
blame on students. There is an understanding of the "we are all in this 
together" aspect of the term, with students having their own high degree of 
stress impacting their role as a student. Many faculty attribute this to the 
move to online instruction, and the lack of interpersonal contact between 
student and teacher. 
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Faculty Voices: 


“Students spend their time on how to cheat or use online resources to do the problems 
for them, rather than trying to learn anything. This is creating a generation of 
incompetent engineers!” - Engineering Faculty 


“We have to rethink evaluation and assessment... My assessments are open book, 
open notes and collaboration — with a focus on ability to solve problems.” - 
Computer and Information Sciences Faculty 


“| would prefer to use lockdown browser and to monitor my students in real time since 
my only choice now is to tell them after the fact that | think they cheated.” - 
Mathematics Faculty 


“| would say the integrity issue is more severe in STEM. Other disciplines can assign 
essays or questions that will have answers unique to each student. That is hard to 
do ina pre-calculus course, for example, unless you create a different assessment 
for each student which is not reasonable from a time perspective.” - Mathematics 
Faculty 


The second-ranked barrier, the "Inability to ensure academic integrity" was 
ranked as most serious by 17% of faculty respondents. As with the issue of 
student engagement, faculty often see this as the result of the shift to online 
teaching, either because less checking possible, or because overwhelmed 
students are just looking to get through the class. 


Faculty Voices: 


“| do worry... how many disadvantaged students we will lose or have lost due to equity 
and financial issues that are a result of the pandemic.” - Biological and Biomedical 
Sciences Faculty 


“I think there is a great opportunity to make education more widely available and 
flexible for students with the new tools and approaches we are using in right now 
due to COVID. | am cautiously optimistic based on this, but | don't see a strong 
future vision from the University.” - Engineering Faculty 


“A major challenge will be equity issues in the broadest sense. How do we reach all of 
those wanting or requiring a STEM education?” - Biological and Biomedical Sciences 
Faculty 


Belief that "Online courses pose equity issues for some of my students" was 
ranked third. Online STEM coursework’s reliance on speedy internet 
connections, capable computer hardware, and specialized lab equipment 
presents unique challenges for members of underserved populations. These 
issues are explored in greater detail later on in this report. 
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Faculty Voices: 


“Our upper level investigative labs are unique and designed to mirror faculty research, 
so pre-made or "canned" online labs do not work for us — although it might be 
possible to use some for basic training.” - Biological and Biomedical Sciences Faculty 


“The primary barrier to effective STEM teaching and learning online is lack of access to 
lab equipment and materials.” - Biological and Biomedical Sciences Faculty 


“lam very concerned about my upcoming semester that involves a lab class. If we 
convert to 100% online | don't know how to conduct experiments remotely. Plus the 
significant time it will take to make this conversion.” - Engineering Faculty 


Respondents believe that almost all barriers they face in teaching STEM 
courses online are the same as those faculty in non-STEM discipline face. The 
only barrier that a majority of STEM faculty feel is unique to their disciplines is 
the need for online labs where 72% felt this was unique to STEM. A further 
47% thought that problems in transfering credits was unique to STEM. No 
other barrier had over 10% who reported it as a unique STEM issue. 


Barrier is unique to online STEM education 

Inadequate online laboratories 

Online courses will not transfer 
Students not technologically prepared 
Lack of technological support 

Support developing or delivering online 
Lack of technology 

Students not sufficiently motivated 
Lower retention rates in online courses 
Inability to ensure academic integrity 
Equity issues for some of my students 
Student internet and computer access 


Institutional support for students 


“| 


g 
g 
: 
3 
§ 
g 
: 
9 
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80% 


STEM Laboratory Coursework 


Faculty Voices: 


“There is a physical technique associated with quite a bit of the chemistry lab work 
that | value. Students need to feel the pipette, move the stop-cock with their hands, 
etc.” - Physical Sciences Faculty 


“My strategy has been "if students can't go to the lab, the lab must go to the 
students". They have acquired basic instrumentation to do hands-on work at home, 
with reimbursements provided by the department.” - Engineering Faculty 


“| think that some laboratory concepts can be taught online, but there really is no good 
digital substitute for physical skills such as dissection and microscopy 
unfortunately.” - Biological and Biomedical Sciences Faculty 


The hands-on experience provided by labs is an important part of many STEM 
courses. Health and safety requirements enacted to curtail the COVID-19 
pandemic have hampered the ability to conduct these in-person elements of 
coursework. The difficulties inherent in conducting meaningful lab exercises 
online have been a recurring item of concern raised by survey respondents. 


A slight majority of all respondents report having a lab associated with one or 
more of their courses. 


Labratory or other practical experience 


m Yes 


m Yes, asa separate 
course 


mNo 
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Status of the laboratory 


Taught completely online 49% 


Taught combination of in person and 40% | 
online 
Taught in person | 23% | 


Other I 4% 
Postponed | 2% 


Canceled % 2% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Due to pandemic-related restrictions, less than a quarter of STEM professors 
are currently teaching their labs entirely in person. The inability to perform in- 
person, hands-on labs has forced some instructors to turn to alternative 
activities online such as simulations, the use of kits, or other laboratory 
experience. 
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Online STEM Laboratory Coursework 


Faculty Voices: 


“| think it is possible to have BETTER laboratory experiences online than in person, but 
it takes a lot of thought, time, and development effort to make that happen. The 
need for lab experiences online to be more self-directed forces a positive change in 
lab experiences in general in engineering.” - Engineering Faculty 


“It takes one to be creative and think out of box to provide a lot of the lab experiences. 
While all those expensive equipment are nice to have and use, we can actually 
teach a lot without them too.” - Engineering Faculty 


STEM faculty are generally optimistic about the potential for online labs, 
despite the great deal of concern expressed about their current state. 


Can an online laboratory be an effective teaching tool 

m Yes 

m Probably, if done 
correctly 

m Perhaps in some cases 


w Unlikely 


mNo 
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Faculty Voices: 


“| think that online labs might be sufficient for lower level chemistry courses, but 
certainly not for organic and above. | can't believe a whole cohort of students are 
going to get through the entire organic series without having touched a piece of 
glassware, run an instrument, or even run a reaction?! Unacceptable.” - Physical 
Sciences Faculty 


“In some ways my students are getting a richer introduction to the lab work as we 
have time that would otherwise be used in waiting for setting up the activity and 
waiting for the organisms to grow.” - Biological and Biomedical Sciences Faculty 


Only a small minority of respondents (18%) currently make use of online labs. 
A slightly smaller number of respondents (14%) are aware of online labs, and 
are considering implementing them for coursework in the future. The largest 
group of respondents (35%) are aware of online labs and online lab practices, 
but feel that these don’t meet their instructional needs. 


Explored any online laboratories or online 
laboratory practices 


Aware but they do not meet my needs 
Currently use them [RES% 
Aware buthave not explored [ih 


Aware and considering for future use 


H Ei Ef 


Not explored these 


No need for an online laboratory I 4% 
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Faculty who do make use of online labs are acquiring them from a varied array 
of sources. Commercial vendors make up the bulk of these, with 54% of 
respondents stating that they acquired their labs from a commercial source. 
Federal and State agencies, by contrast, provide online lab resources to just 
9% of respondents. 


Online lab source 


Colleague recommendations 58% 


Design myself 


Online repositories 
Professional societies 
Institutional recommendation 27% 


Federal or State Agencies [| 9% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Recommendations also play an important role in choosing an online lab. An 
institutional recommendation played a role in selecting an online lab for 27% 
of respondents, while the majority of respondents, at 58%, chose an online 
lab based on a colleague’s recommendation. 
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Equity Issues 
Faculty Voices: 


“There is anew urgency for quality, diverse, and inclusive STEM education that gives us 
a fuel and opportunity to make change happen. | am very optimistic for change to 
happen, but pessimistic about Higher Education's upper administration's willingness 
to follow through.” - Physical Sciences Faculty 


“Diversity and inclusion are extremely important for the future of STEAM and our 
youth. The frustration that our white students are feeling during the pandemic is a 
mere "tip of the iceberg" for the constant battle our marginalized students deal 
with.” - Biological and Biomedical Sciences Faculty 


Respondents report that the transition to remote learning has had a 
deleterious effect on the equity of educational opportunities in several 
segments of their student population. 


Equity changes for teaching online 


Collaboration among students [ly 28% 61% 
Access to education for under-served students 16% 39% 45% 
Student access to support services 20% 37% 43% 
Participation by shy or introverted students 22% 35% 43% 
Ability of students with physical disabilities to succeed 24% 44 bah 4 
Discrimination by race, age, ethnicity, or sexual orientation [R}%4 81% 9% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


mBetter mNochange m Worse 


Sixty-one percent of respondents believe that collaboration among students 
has grown worse, while 10% reported an improvement. Forty-three percent of 
respondents saw a decline in participation by shy or introverted students, 
while 22% saw an improvement. Eighty-one percent of respondents reported 
that they thought moving courses online would not impact levels of 
discrimination by race, age, ethnicity, or sexual orientation. This does not 
mean that they believed that discrimination did not exist, only that moving a 
course online, by itself, would not reduce or add discriminatory actions. 
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Next Steps 


This survey, Teaching Online: STEM Education in the Time of COVID, provides 
insights into the experiences of STEM faculty who underwent a hurried 
transition into remote education, and how that experience informed their 
opinions of online education’s effectiveness. As with all research, the survey 
raises additional questions, suggesting additional research and future work. 


One of the key questions to explore is the source and reliability of information 
used by faculty to guide them in addressing their perceived issues with online 
labs. Colleagues and vendors were a primary source of information on tools to 
facilitate remote education for faculty, while professional societies and 
institutional support services played a lesser role. Does this present an 
opportunity for these groups to expand their roles within the STEM fields? 
What awareness do faculty have of peer-reviewed activities within national 
Open Educational Resource repositories? How can professional societies 
better assist STEM faculty in developing and evaluating strategies to teach 
online effectively? 


Another area of interest is the perception that labs represent a unique 
challenge within the STEM disciplines. Hands-on or skills-based activities occur 
in non-STEM disciplines. Could online practices within fields such as art, 
dance, or music be adopted into STEM fields, to facilitate a hands-on 
experience? What core activities across these disciplines meet the learning 
objectives associated with hands-on experience? 


Finally, COVID-19 has caused a significant shift in how faculty view online 
education within the STEM fields. As with any sudden change, the experience 
of faculty teaching online has been varied — some good, and some bad. 
Overall, results suggest that faculty are cautiously optimistic about the future 
of online STEM education. This optimism suggests that individuals and 
institutions will use their experience in response to the COVID-19 pandemic as 
an opportunity to reassess how STEM education is currently delivered, and 
will be more open to new approaches that incorporate online education and 
digital learning into their curricula. Understanding how to better develop, 
deploy, and evaluate the tools and strategies faculty could employ for 
successful outcomes in online learning could prove crucial in the years to 
come. 
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METHODOLOGY 


This report's data comes from survey results using a national sample of faculty 
and designed to represent the overall range of all STEM teaching faculty in 
U.S. higher education. A multi-stage selection process was used for creating 
the stratified samples. 


The process began by obtaining data from a commercial source, Market Data 
Retrieval, which has over one and a half million faculty records and claims that 
its records represent 93% of all teaching faculty. All faculty who taught at least 
one course in a STEM field were selected for this first stage. Individuals were 
then randomly selected from the master list in proportion to the number 
contained in each Carnegie Classification, to produce a second-stage selection 
of STEM teaching faculty. This list was then checked against opt-out lists, as 
well as for non-functioning email addresses. 


A total of 869 faculty responded to a sufficient number of questions to be 
included in the analysis, representing the full range of higher education 
institutions (two-year, four-year, all Carnegie classifications, and public, private 
nonprofit, and for-profit), and the complete range of faculty (full- and part- 
time, tenured or not, and all disciplines). 


Institutional descriptive data come from the National Center for Educational 
Statistics’ Integrated Postsecondary Education Data System (IPEDS) database. 
After the data were compiled and merged with the IPEDS database, 
respondents and nonrespondents were compared to ensure that the survey 
results reflected the characteristics of the entire population of schools. The 
responses were compared for 35 unique categories based on the 2015 
Carnegie Classification of Institutions of Higher Education. 
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Participants 


Responses were collected from a total of 896 higher education faculty. 
Participants represented 49 states and the District of Columbia. The largest 
group of respondents (62%) were from four-year public institutions, while 29% 
were from four-year private institutions, and 9% from two-year institutions. 


Sector of institution 
m Public, 4-year 
@ Private not-for-profit, 4- 


year 
m Public, 2-year 


Participant institutional affiliation was matched to the IPEDS database, to 
retrieve institutional characteristic data allowing for analyses to be conducted 
by type of institution. The IPEDS data show that 84% of the respondents were 
from an institution with graduate-level offerings, 60% of which had a 
"research" Carnegie classification. 


Carnegie classification Graduate offering 
m Research mw Graduate degree 
mw Masters offering 
@ Associates 
w No graduate 
@ Baccalaureate offering 


w Specialized 
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The responding STEM faculty are primarily from the larger institutions. A 
majority are from institutions with more than 10,000 total enrollments, with a 
full 40% at institutions with 20,000 or more students. Only 1% are at 
institutions with less than 1,000 total students. 


Institution size category 


mw Under 1,000 
m 1,000 -4,999 
@ 5,000 -9,999 
m 10,000 - 19,999 


@ 20,000 and above 


Procedures 


All data were checked for completeness, missing values, or erroneous codes. 
All responses entered as “other” were reviewed to determine if they should 
also be coded as one of the fixed responses. Respondents could skip any 
question, though very few respondents did; all surveys where less than three- 
quarters of the eligible questions were not completed were deleted from the 
analysis. 


Respondents were asked if they would provide permission for their open- 
ended responses to be quoted in this report; 76% of respondents agreed. All 
open-ended responses were checked to ensure that no personally identifiable 
information was contained, and only those respondents who provided explicit 
permission to be quoted are contained in this report. 
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Using the Report 


Bay View Analytics wants our reports used as widely as possible, so we try to 
make it as easy as possible for you to use our materials. All Bay View Analytics 
publications carry a Creative Commons Attribution 4.0 International (CC BY 


4.0) license. 
(Omom 


Under this license, everyone is free to: 


e Share — copy and redistribute the material in any medium or 
format. 


e Adapt — remix, transform, and build upon the material for any 
purpose, even commercially. 


Under the following terms: 


e Attribution — You must give appropriate credit, provide a link 
to the license, and indicate if changes were made. You may do 
so in any reasonable manner, but not in any way that suggests 
the licensor endorses you or your use. 


There is NO requirement to ask permission to use Bay View Analytics 
publications under the CC BY 4.0 license. That said, we always like to hear how 
you are using our material. 


We encourage the broadest use of our reports. You may reproduce the report 
in any media (e.g., print or electronic distribution) you wish and distribute it in 
any way you wish. We ask that if you are distributing a full PDF version of this 
report you link to www.bayviewanalytics.com rather than hosting it ona 
different site, as this allows us to track the level of interest in our reports. 


We understand that some publications may require a statement explicitly 
stating that the intended use is approved. If a copy of this page is not 
sufficient for such purposes, please let us know, and we will provide the 
required documentation. 
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The National Survey on the State of Online STEM Education 


Origin 


The National Survey on the State of Online STEM Education (NSSOSE) 
represents the work of many individuals and organizations striving to better 
understand the opportunities, barriers, and potential of online education 
within the STEM disciplines. Developed through community input, the survey 
was designed to benchmark the current state of online STEM education and to 
lay the foundation for further growth in online STEM education. The survey 
results highlight the need for additional research into the evaluation and 
design of effective online tools, strategies, and practices, particularly around 
online laboratories. 


The motivation for a national survey was originally based on an assumption 
that STEM educators were reluctant to incorporate online strategies and tools 
into the STEM curriculum. This assumption was supported by comments made 
in 2007 by Myles Boylan, then of the National Science Foundation who stated: 
“In almost every discipline, | could point to a variety of really effective, 
wonderful sets of instructional materials and instructional practices, and say 
that if we could magically click our fingers and get everybody using them, 
there would be a huge improvement in undergraduate education that would 
happen instantaneously, but we're nowhere near that.”? As true today as it 
was in 2007, understanding the factors that limit the adoption of instructional 
materials and practices, especially in online STEM education, is critical. 


The first steps in developing the NSSOSE came from recommendations put 
forward by STEM educators, administrators, and STEM-based educational 
companies participating in a STEM focus group as part of OLC’s 2019 National 
Innovate conference. 


Through a moderated focus group, participants highlighted the need to better 
understand the opportunities available to educators and the potential to 
expand online STEM education by developing a community of practice. A key 
outcome was the acknowledgment of significant data gaps in understanding 
the awareness, effectiveness and acceptance of online STEM education within 
the broader STEM community. To address this data gap, the focus group 
recommended developing a national survey to benchmark the state of online 
STEM education. 


3 Myles G. Boylan, NSF, Chronicle of Higher Education, The Tough Road to Better Science Teaching, August 3, 2007 
nnn nnn ________________________ _i_____ ____§__] 
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To accomplish this, educators from a number of organizations began the work 
of developing the survey. Recommendations for the types of questions that 
would provide the greatest amount of insight into the state of online STEM 
education were solicited from the broader community throughout the year. 
Additional activities included trial surveys, and input from working groups as 
part of the OLC Accelerate 2019 STEM Research Summit and OLC IDEATE 2020 
conferences. 


The survey was scheduled for national release in March of 2020. The national 
emergency declaration caused by the COVID-19 pandemic stalled its release. 
The sudden shift to online instruction in response to the pandemic provided 
the working group with the opportunity to re-design the survey, to capture 
this experience. Many STEM faculty had little or no previous experience with 
online education, and support provided by institutions was variable. The non- 
voluntary shift to online instruction and the resulting experience was 
important to capture to better understand the overall perceptions of online 
education by STEM faculty. The re-designed survey was released in the fall of 
2020 to a national sampling of STEM educators. 


In addition to input from conference participants and other volunteers, the 
survey was developed by a STEM working group in consultation with a STEM 
advisory group. The Working Group was charged with the development and 
management of the survey, including analyzing survey results, developing the 
final report, and disseminating results. The Advisory Committee and 
Reviewers provided feedback, advice, expertise, and review of the STEM 
survey and final report. 
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STEM Working Group and Affiliations 
e Devon Cancilla, Ph.D., Co-chair, Cancilla Digital Education 
e Jeff Seaman, Ph.D., Co-chair, Bay View Analytics 
e Kate Lee-McCarthy, MA, Director of Grants, OLC 
e Lynette O’Keefe, Ph.D., Director of Research, OLC 


e Isabel Allen, Ph.D., Professor of Biostatistics & Epidemiology, 
UCSF 


Advisory Committee and Reviewers 


e John Whitmer, Ph.D., Senior Fellow, Data Science, Federation 
of American Scientists 


e Melissa Csikari, Program Manager, HHMI Biointeractive 


e Shannon McGurk, Director, Distance Learning, Carolina 
Distance Learning 


e Siobhan Paul, Director of Global Marketing, DigitalEd 


e Justin T. Dellinger, Associate Director, LINK Research Lab, 
Center for Research on Teaching and Learning Excellence, U of 
Texas Arlington, Digital Learning Research Network, Every 
Learner Everywhere 


e Chaohua Ou, Assistant Director, Learning & Technology 
Initiatives, Center for Teaching and Learning, Georgia Institute 
of Technology 
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SPONSORS 


S. l eo) hee’ ze https://www.digitaled.com/ 


Nea 
tae fa lap. a https://www.biointeractive.org/ 


BiolInteractive 


CA R() LI iN fy https://www.carolinadistancelearning.com/ 


DISTANCE LEARNING 


CVE lea Mer https://www.everylearnereverywhere.org/ 
everywhere 
OL 


ONLINE LEARNING 
COUN FORT IVR 


https://onlinelearningconsortium.org/ 
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QUESTIONNAIRE 


State of Online STEM Education: National Online STEM Education Survey 


The Online Learning Consortium (OLC) in partnership with Every Learner Everywhere, 
HHMI BioInteractive, DigitalEd, and Carolina Distance Learning and, in collaboration with 
Bay View Analytics, is conducting a National Survey on the State of Online STEM 
Education. This important benchmarking survey will provide critical information on both the 
opportunities and barriers of online STEM education. 


For the purposes of this survey the term STEM education refers to all teaching and learning in 
the fields of science, technology, engineering, and mathematics. 


Please tell us a bit about yourself. Note: This information is used only to classify the survey 
responses. No individual-level data will be released. Information that you provide in this 
survey will not be used to target you for any marketing. 
Please indicate your adjunct status. 

Adjunct 

Not Adjunct 


Please indicate the number of years you have been teaching. 
1 to 10 
11 to 20 
21 to 30 
More than 30 


Gender 
Male 
Female 
Non-binary 
Other 


Prefer not to answer 
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Please indicate discipline of your home department. 
Aerospace, Aeronautical and Astronautical Engineering 
Applied Mathematics 
Astronomy and Astrophysics 
Atmospheric Sciences and Meteorology 
Biochemical Engineering 
Biochemistry, Biophysics and Molecular Biology 
Biological and Biomedical Sciences, Other 
Biology, General 
Biomathematics, Bioinformatics, and Computational Biology 
Biomedical/Medical Engineering 
Chemical Engineering 
Chemistry 
Civil Engineering 
Computer and Information Sciences 
Computer Engineering 
Electrical, Electronics and Communications Engineering 
Engineering Technologies/Technicians 
Engineering not listed elsewhere 
Materials Sciences 
Mathematics 
Mechanical Engineering 
Nanotechnology 
Neurobiology and Neurosciences 
Physical Sciences not listed elsewhere 
Physics 
Statistics 
Other 
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EXPERIENCE WITH ONLINE AND BLENDED/HYBRID COURSES 


What is your experience with online or blended/hybrid courses prior to the pandemic? 
Please check all that apply. 


Online Course: A course in which all, or virtually all, the content is delivered online. 
Typically have no face-to-face class meetings (with the possible exception of proctored 
exams). 

Blended/Hybrid Course: A course where sufficient content is delivered online to create a 
reduction in the number of face-to-face class meetings. 


I have taken an online or blended/hybrid course 
I taught an online course 


I (alone or with others) developed, substantially modified, or converted a face-to-face 
course to online or blended/hybrid 


Other experience with online or blended/hybrid courses 


None of the above 


What is your experience with online or blended/hybrid courses since the start of the 
pandemic? Please check all that apply. 


I have taken an online or blended/hybrid course 
I converted a face-to-face course to emergency remote learning 


I (alone or with others) developed, substantially modified, or converted a face-to-face 
course to online or blended/hybrid 


Other experience with online or blended/hybrid courses 


None of the above 
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COURSES BEING TAUGHT 


How many STEM courses are you teaching this current fall term? 
None 
1 
2 
2 


4 or more 
At what level are you teaching? 
Undergraduate level 


Graduate level 


Other 


How are your courses delivered? 
Face-to-face 
Blended/Hybrid 
Online 


Other 


Do any of your current courses require a laboratory or practical experience (e.g. field work, 
clinical experience, studio)? 


Yes, there are labs or other practical experience that are part of my courses 
Yes, there are labs or other practical experience, but they are part of a separate course 


No, there are no labs 
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LABORATORY, STUDIO, OR PRACTICAL EXPERIENCE 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


What elements are included in the laboratory or practical experience for your courses? 
Visits to locations or field work remote from campus 
Observation or interaction with subjects or patients 
Hands-on use of engineering, mechanical equipment, or scientific instrumentation 
Hands-on use of electronic or computer equipment 
Studio 
Dissection or other exploration of specimens 


Other 


What is the status of the laboratory portions of your courses? 
It is being postponed until it can be taught in person 
It has been canceled 
It is being taught in person 
It is being taught using a combination of in person and online 
It is being taught completely online 


Other 
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BARRIERS TO ONLINE STEM EDUCATION 


In your personal opinion what would be (are) significant barriers to teaching your courses 
online? Please select all that apply. 


There are no significant barriers 

Inadequate or non-existent online laboratories 

Need for support from my institution in developing or delivering the online course 
Students are not sufficiently motivated to succeed in online courses 
Students are not technologically prepared to succeed in online courses. 
Inability to ensure academic integrity 

Lack of internet and computer access for online students 

Lack of technology at my institution 

Lack of technological support at my institution 

Lack of institutional support for online students 

Online courses will not transfer 

Lower student retention rates in online courses 

Online courses pose equity issues for some of my students 


Other 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


Why do you think the transferability of online STEM courses is a barrier to developing these 
types of online courses? Please select all that apply. 


Inability to match course sequences across institutions 
Course learning outcomes are not aligned across institutions 
Credit hour allocations for laboratories differ 

Skill outcomes for laboratories differ 


Online courses do not have the rigor required for majors 
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Other 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


Only Display Selected Choices from "In your personal opinion what would be (are) 
significant barriers to teaching your courses online?" 


Are these barriers unique for online STEM courses or do they impact all online courses? 


STEM-only INI Koya btits 
issue courses 


There are no significant barriers 


Inadequate or non-existent online laboratories 


Need for support from my institution in developing or delivering the 
online course 


Students are not sufficiently motivated to succeed in online courses 
Students are not technologically prepared to succeed in online courses. 
Inability to ensure academic integrity 

Lack of internet and computer access for online students 

Lack of technology at my institution 

Lack of technological support at my institution 

Lack of institutional support for online students 

Online courses will not transfer 

Lower student retention rates in online courses 

Online courses pose equity issues for some of my students 


Other 
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The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


Only Display Selected Choices from "In your personal opinion what would be (are) 
significant barriers to teaching your courses online?" 


Which of the barriers that you selected do you believe is the most serious? 
There are no significant barriers 
Inadequate or non-existent online laboratories 
Need for support from my institution in developing or delivering the online course 
Students are not sufficiently motivated to succeed in online courses 
Students are not technologically prepared to succeed in online courses. 
Inability to ensure academic integrity 
Lack of internet and computer access for online students 
Lack of technology at my institution 
Lack of technological support at my institution 
Lack of institutional support for online students 
Online courses will not transfer 
Lower student retention rates in online courses 
Online courses pose equity issues for some of my students 
Other 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


Do you believe that it is possible for an online laboratory to be an effective teaching tool? 
Yes 
Probably, if done correctly 
Perhaps in some cases 
Unlikely 
No 
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The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


Have you explored any online laboratories or online laboratory practices? 
I have no need for an online laboratory 
I have not explored these 
I am aware that these exist, but have not explored them for my own use 
I am aware that these exist, but found that they do not meet my needs 
I am aware that these exist and I am considered them for future use 


I currently use an online laboratory kit, simulation, or exercise 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


What sources of online laboratories or online laboratory practices have you explored or are 
planning to explore? 


Those I have design myself 
Commercial vendors 

Professional societies 

Federal or State Agencies 
Recommendations from colleagues 


Recommendations from institutional teaching and learning or instructional design 
teams 


Online repositories (such as the National Science Digital Library (NSDL), PhET, or 
Merlot). 


Other 


[OPTIONAL] We welcome any thoughts you may have on the use of laboratories, studios or 
practical experiences in online STEM courses. 
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FUTURE OF STEM EDUCATION 


Based on your experiences to date, your view of your institution’s past and future prospects, 
and your overall opinion of higher education in general, how optimistic or pessimistic are 
you... 

Optimistic Neutral Pessimistic | Unsure 


About the future of STEM education? a 
About your personal role in higher education? a 


[OPTIONAL] Please let us know why you provided the ratings that you did. 


Equity/Accessibility 

Have the pandemic-induced changes at your institution improved access and equity issues for 
our students or made them worse? 

Student access to support services (e.g., peer tutoring, academic coaching). 

Access to education for under-served students. 

Participation by shy or introverted students. 

Collaboration among students. 

Discrimination by race, age, ethnicity, or sexual orientation. 


Ability of students with physical disabilities to succeed. 
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Every Learner Everywhere is a network of diverse educational organizations that work 
collaboratively to improve student outcomes. Every Learner Everywhere is committed to 
understanding issues of equity and diversity in online STEM education. Please take a moment 
to answer three questions around race and ethnicity. This section is voluntary and will provide 
additional context to better understand the overall results from the survey. 


You may skip all of the following questions by pressing the arrow button at the bottom of this 
page. 


What race do you primarily identify as? 


Do you have a secondary race that you identify as? 
Yes 
No 


The following question was only displayed to the respondent if they provided a qualifying 
answer to a previous question 


What is it? 


(Optional) Do you have any final thoughts related to teaching STEM courses online? 


Thank you for your time in completing this survey. All respondents will receive a copy of the 
project report when it is released. 


May we quote your responses? Responses will be identified only by the subject area of the 
faculty member (e.g., Chemistry Faculty, Mathematics Faculty, etc.). 

Yes 

No 
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